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ABSTRACT 

The use of a general purpose simulator (GPS) to teach 
Air Force technicians diagnostic and repair procedures for 
specialized aircraft radar systems is described. An EC II simulator 
manufactured by Educational Computer Corporation vas adapted to 
resemble the actual configuration technicians vould encounter in the 
field. Data acguired in the evaluation included: (1) the experience 
level of the participating personnel, (2) learning patterns using the 
simulator, (3) evaluation of the training potential, and (4) 
attitudes and acceptance. All personnel exposed to the GPS stated 
that learning vas easy, and tests given the technicians after their 
training sessions indicated a high level of achievement. The system 
vas generally found to be acceptable to trainees, although some 
participants indicated a preference for more traditional teaching 
methods. The cost of using the GPS vas found to be less than 10 
percent of the costs of an actual eguipment trainer. (DGC) 
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FIELD EVALUATION OF A GENERAL PURPOSE SIMULATOR 

INTI^DDUCnON 

People leain by doing. Demonstrations can be useful in orienting 
a student to selected tasks. However, active practice of the task being 
learned msc be provided systematically to the student in the design 
of instruction. Feedback to the learner also should be an integral 
part of this practice. One means of providing both practice and feed- 
back opportunities in the learning process is simulation. 

Simulation, as a method of instruction, is effective for teaching 
many tasks and skills in technical training, A wide variety of pro- 
cedural sequences perceptual -motor skills, identifications, conceptual 
tasks, and team functions have been effectively learned through the 
use of simulation. The approach of systematically abstracting and 
partially duplicating tasks, activities, or operations provides transfer 
of training from a synthetic environment to a real environment. Simula- 
tion allows student involvement in learning, paced to the needs of each 
individual. Both practice opportunities and forms of feedback usually 
not available when using the actual equipment or when operating in the 
real world are also provided. 

Simulators represent a real situation in which tasks are performed 
or operations are carried out. They omit, however, selected parts of 
the actual operatiai which are psychologically unimportant to the task 
or operation, parts which are dangerous, parts which are expensive, and 
hopefully they also omit some of the unpredictabilities of the real 
world. Simulators provide the learner with predetermined levels of 
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control over the task or operation thus allowing controlled practice on 
representative or critical aspects of the selected tasks or operations. 

For training effectiveness the simulator must provide psychological 
realism (see Miller, 1954). The student must receive information identical 
to job inputs or inputs identifiably representative of job information 
inputs must be provided. The meter reading^ the trouble lights, the 
test results, and other infomation should be like the actual job task 
performance. The student must be able to exercise some level of control 
over the system, typically cued by the system inputs. The student should 
be able to correct a malfunction, for exanple. The consequences of the 
students interaction with the system must be represented. Baker and 
Wamick (1970) state that operational similarity and motivational similarity 
must be incorporated into the simulation. 

Simulation has training advantages over real wo^-ld operation. 
Baker and Wamick (1970) list six techniques to enhance transfer of 
training. Application of these techniques mak^s the training situation 
physically dissimilar to the actual situations. These six conditions 
are: 

(1) Provide augemented feedback, i.e., extra knowledge of results 
during instruction. 

(2) Increase the number and frequency of crises, conflicts, equip- 
ment breakdowns, and e^nergencies. 

(3) Reduce the ope rationed time for certain events, so as to 
incrc^ase the amount of practice on critical skills. 

(4) In the total perfomance behavior, vary the part- task sequence. 



because snail amounts of practice on several similar tasks promote mre 
learning than large blocks of practice on a single task. 

(5) Provide guidance and stimulus support in the eariy or initial 
stages of learning* 

(6) During training, vary the progression of difficult/ levels; 
a progressively easy-to-difficult procedure facilitates transfer of 
training. 

IVhile actual equipment trainers or simulators are iised for demon- 
strations, they ;;re not widely available for student use in technical 
training for a variety of reasons, '^s results in a lack of hands-on 
practice or learner involvement fcr the students on job related skills. 
Unavailability of this training resource may result in lower level of 
training which uses more classroom instruction time. It may take more 
time to acconplish less. Ulien not provided job related skill training 
in school, the technical training graduate requires more on-the-job 
training. This reduces the job time available to the field unit from 
both the student and the person training him. IVhen available, actual 
equipment simulations typically are used as demonstration devices and 
procedures trainers. Job skills such as troubleshooting and maintenance 
often are not trained using this equipment due to (1) the difficulties 
in inserting malfunctions, and (2) the possibility of damage by inexperi- 
enced students. 

One option for increasing hands-on training recently explored under 
Air Force contract by Applied Science Associates is the use of general 
purpose simulation. This type of simulation, intended to provide an 



economical approach to training, provides a standard frame with a control 
system to which modules are added to make specific pre-programmed simula- 
tions. Interchangeable faceplates associated with pro^irammable electronic 
circuitry provide the required modular flexibility. Thus simple changes 
in the faceplate and programming of the electronics permit the general 
Durpose simulator to provide a wide range of pre-planned simulations. 

Currently in Meadville Pennsylvania, an experimental educational 
project in vocational guidance uses general purpose simulation to acquaint 
sixth to ninth grade students with skills and job tasks in the automotive, 
ri;dio-TV, and major appliance repair areas. Programmed exploratory 
lessons are provided on audio tape which integrate hands-on training 
with the simulator. However, the most use of programmable simulators 
is in military training. 

A general purpose simulator (GPS) marketed as the EC 11 was pro- 
cured from the Educational Computer Corporation. Following special 
factory training and an analysis of the course learning requirements, 
a faceplate (simulation display panel) and a sequence of slides were 
designed so as to simulate the APQ 126 Radar System which is associated 
with the \7d Aircraft. The computer program implementing the face- 
plate and selected slides was developed by Educational Conputer Corpora- 
tion and procured from them. The finished product is pictured in Figure 1. 



Insert Figure 1 about here 
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Evaluation of the resulting simulator was perfomed in several 
ways. Cost conparisons, effectiveness in learning, learaer attitudes, 
^id an analysis of design difficulty were performed in a selected 
Avioriics course and reported in McGuirk and Pieper (197^). Results 
showed that the initial cost for an operational GPS was less than ten 
percent that of the equivalent actual equipment trainer (AET). Nbre- 
over, training in techniques of malfunction isolation was not possible 
using the conparable AET. Training on both the GPS and the AET permitted 
student learning to the criterion performance level on normal operational 
procedures. No interference occurred when transferring either from the 
GPS to the AET or the AET to the GPS. Practice and feedback on the 
isolation of malfunctions was provided only on the GPS, a capability 
previously unavailable to the course of instruction. The GPS provided 
significant learning opportunities not previously available, nor readily 
feasible using other available modes. 

A field evaluation using field personnel for instructors was requested 
by the Air National Guard Bureau to detemine if general purpose simula- 
tion would assist in their on-the-job training program. While the GPS 
was shown cost-effective and feasible in a technical training resident 
school environment, its use in job training on the flight line had not 
been evaluated. The purpose of the present r**X.'mi study was to evaluate 
the usability, effectiveness, and acceptability (by instructors and 
students) of general purpose simulation. The simulation of the APQ 126 
Radar System previously developed for use in an Avionics Course in a 
formal Technical Training School was used. This simulation^ based on the 
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school rcquireinents, enabled training in nomal operational procedures 
and malfuTjCtion isolation at the apprentice level on the APQ 126 Radar 
System. 

Training was conducted by National Guard personnel. An experienced 
journeyman level Weapons Systems Mechanic on the A7D aircraft was trained 
in the use of the APQ 126 simulation program on the GPS. Later an 
apprentice level mechanic with previous experience in teaching school 
was trained so as to perform as r instructor. 

The joumeyr.an level technician oriented all the Weapons Control 
Shop mechanics to the GPS at Buckley Field. Orientation consisted of 
showing the mechanics how to operate the simulator and providing practice 
on isolating some predetermined malfunctions. Then the mechanics were 
tested in malfunction isolation. A questionnaire (see /appendix A) was 
administered before and after working with the simulator. This question- 
naire was used to determine previous experience, learning, a comparative 
evaluation of training potential, attitudes, and acceptance. The time 
and nunber of replacements for malfunc tion isolation also were recorded 
by each technician. Since all these technicians were skilled in trouble- 
shooting this exercise provided an indicator of acceptability and per- 
ceived usefulness of the simulator. Also provided was an understanding 
of seme of the capabilities of this approach to training. 

On a different day, the three reserve Air National Guard personnel 
were trained by the apprentice level technician. For the first hour, 
all three were trained together in normal system operation. An additional 



hour and one half instruction was provided to each individually. Following 
training, each student was tested in malfunction isolation. Questionnaires 
were administered before and after training. 

On the following day, three journeyman level technicians from the 
Avionics ^\crospace Ground Equipment Repair Shop were cross -trained using 
the GPS. The apprentice level technician conducted the training in the 
same manner as previously described for the reserve personnel training. 
Data was thus acquired from eleven qualified (journeyman level Weapon's 
System Mechanics on the A7Q Aircraft) personnel and six unqualified 
personnel. Participation was 100%. 
RESULTS 

Data acquired in the field evaluation included (1) the experience 
level of the participating personnel, (2) learning, (3) evaluation of 
training potential, and (4) attitudes and acceptance. 
Experience . Nino of the eleven full time National Guara personnel in 
the We^ons Control Shop had between four • experience. 

Median level experience for the eleven qualiiiea personnel was years. 
All were familiar with the A\/APQ 126 Radar System and reported themselves 
qualified to troubleshoot this system. 

Distribution of skill level was as follows: Supervisory- level three 
men, journeyman -level seven men, apprentice- level one man. Six personnel 
had worked on othvir radar systems, while five had no experience on any 
other radar system. It ^as reported to the cxperimentor that other 
radar experience v/ould provide only minimal transfer to the operation 
of this system. The primary mode of initial training received on the 
8 
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,\PQ 126 Radar S/btcm by qualified personnel was reported as: Actual 
equipment trainer-6 persons. Equipment as installed on aircraft-4 persons, 
Lecture and technical order- 1 person. 

The three reserve personnel had bet^^een 2h and 6^ years experience. 
One had worked on the A7D aircraft under supervision, and two had begun 
informal training on the APQ 126 Radar System. All had limited experience 
on another radar system. None was qualified to troubleshoot this radar 
system. 

r 

The three Avionics Aerospace Ground Equipment Repair Shop personnel 
each carried a journeyman level skill in their specialty and ranged from 
5i to 5 years experience. One reported sane familiarity (and no training) 
with the APQ 126 Radar System and two had experience on other radar 
systems. None were qualified to troubleshoot the APQ 126 Radar System. 
Learning . The apprentice level -training program, which the GPS implemented, 
Was intended to teach: 

(1) The purpose of the controls, 

(2) The interpretation of normal versus malfuiiction operation, 

(3) The performance of system self -checks, and 

(4) The exercise of malfunction isolation. 

bach person was asked to evaluate either if the GPS could effectively 
provide the desired instruction or if they were able to do what the 
program was intended to teach as a result of the instruction. Table 1 
shows the results which include the performance test scores on malfunction 
isolation. To be noted is that the isolation of malfunctions is not 
possible without achieving the three enabling goals. Thus the scores 
9 
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which indicate a self report on learning are substantiated by the per- 
formance test. Two five- level personnel felt that appropriate system 



Insert Table 1 about here 
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self checks were not adequately covered by the specific program evaluated 
and one felt that malfunction isolation was not adequate for field use 
in the three -level program. 

Hvaluation of Training Potential . Only the opinions of the qualified 
personnel are summarized since their duties could 'all for them to 
provide training. They also have enough job experience to realize which 
aspects of the job would require training. These experienced technicians 
felt that if they were a training person that the fastest primary mode 
of instruction would be: 

General Purpose Simulator - 5 persons 
Installed equipment - 4 persons 
Actual equipment trainer - 2 persons 
Experienced technicians felt that the most effective mode of learning 
would be: 

Installed equipment - 8 persons 
Actual equipment trainer - 3 persons 
Nkxie of instruction making learning the easiest was felt by experienced 
technicians to be: 

G^neral Purpose Simulator - 6 persons 
Installed equipwnent - 3 persons 
Actual equipment trainer - 1 person 
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Attitudes and acceptance . All personnel rated the learning on the GPS 
as easy, while fourteen of seventeen liked the GPS application to learning. 
^UI but one of the experienced technicians recommended the use of a simu- 
lator such as the GPS of the APQ 126 for use in training of the APQ 126 
I^dar System and other systems to be trained. Other specific recommenda- 
tions concerning specific applications in Avionics systems training were 
made by various personnel. 
DISCUSSION 

The evaluation of the APQ 126 Radar System simulation on the GPS 
conducted at Buckley Field was designed to determine usability, effective- 
ness, and acceptance in a job environment. 

All personnel exposed to the GPS stated that learning was easy. No 
difficulties were encountered in the use of the GPS, The training program 
on the APQ 126 was conducted by the least experienced of the Weapons 
Control Shop mechanics--aMapprentice-level person, only 2h hours 
for each student, satisfactory training was completed in both nonnal 
and malfunction isolation mode for all students. 

The goals which the simulator was designed to achieve were accom- 
plished by all personnel. Not only were the students asked whether they 
had learned specific tasks or operations, a performance test also showed 
a high level of achievement using the GPS. Operational similarity was 
adequately achieved. Of particular note is the programmed capability 
provided by the simulator enabling training in the isolation of selected 
malfunctions. Experienced mechanics given the write-up (reported malfunc- 
tion) averaged less than two minutes to correct each malfunction in the 
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simulation. Average time on the flight line would average over an hour 
to correct each malfunction, and probably slightly longer when training 
is also being accomplished. This time compression enables more practice 
developing the concepts involved in the isolation of malfunctions, ITie 
big picture is made clearer. 

Acceptance of the GPS provides an indication of motivational simi- 
larity--a similarity in feeling or attitude on the part of the student 
between the real equipment and the simulator. Experienced mechanics 
generally indicated that training could be performed faster and easier 
on the GPS, This would indicate acceptance. The GPS was recommended 
for training by most of the personnel. This also indicates acceptance, 
Mien directly queried most personnel liked using the APQ 126 Radar System 
simulation on the GPS, However, a preference for teaching the way in 
which you yourself liave been taught is shown by the selection by several 
personnel of installed equipment as the most effective mode of learning. 

Data collected from field use of a GPS indicated that psychological 
realism was economically captured in a simulation of the APQ 126 Radar 
System, Incorporated were satisfactory levels of operation similarity 
and motivational similarity to the real equipment. As previously stated, 
the simulation, specifically designed for training, provided many advantages 
not obtainable when using the real equipment, at far less cost than either 
the real equipment or an actual equipment trainer, 

IVhile simulation has been shown as an effective way of training for 
a wide variety of specific operations or tasks, the principles upon which 
the successes have been based are not clear. In fact, our knowledge of 
13 



ERLC 



15 



tlie psycliological principle of the traisfe * of training is incomplete. 
Xo predictive body of knowledge is available which will ensure the adequate 
design of a simulator for effective training. The specific goals of the 
training program when used to direct the design of a simulator, generally 
result in a usable simulator, that is, a simulator which provides the 
required training. Factors which when manipulated, can reduce cost and 
increase effectiveness have not been identified prior tp a simulation 
design. Thus, requirements for heuristic research as here presented 
continues. 

One major difficulty, widely recognized by training technologists 
concerning simulation, is the issue of realism. Engineering fidelity 
aiid physical realism are not necessarily incorporated into a GPS. A fixed 
size face plate and a series of selected images and meter readings limit 
the possible visual inputs to the learner. However a visual representation 
can be provided which could not be perceived from any position on actual 
equipment, 'lliis enables the student to form a dynamic cognitive struc- 
turing of the overall situation; that is he can learn to put it all 
together. Typically other related units are not attached to the GPS, 
although it has been done when deemed necessary. Denenberg (1954) first 
showed that physical realism in a tank hull trainer may not be necessary 
in providing necessary and adequate transfer to the job. IVhile no 
generalizable simulation studies have been reported, similar results to 
Denenberg 's study have been reported for a wide variety of simulators. 
ITie visual materials used and selected for training must provide the 
necessary inputs to enable the cuing of appropriate task performances. 
14 



Physical realism provides no assurance that useful information will be 
seen, learned, or remembered. For example physical reality may be too 
cor.plex for i beginning learner to make the appropriate discriminations, 
associations, and generalizations so as to meet the training require- 
ments. Representations of reality frequently must be simplified and 
stylized when learning efficiency is desired, (see Travers and others 
1964) . 

It has been customary in procuring new simulators to require 
engineering fidelity; that is, the simulator is required to function as 
nearly as possible in the way in which the real equipment functions, 
llie false assumption is that better transfer can be directly associated 
to more realistic representations. Not only is this assumption ex-pensive, 
it may also be detrimental to training. An operational item of equip- 
ment may not be designed for operating in ways which would provide the 
most effective training. For example, a landing gear hydraulic activator 
may be quite well desi.ijied for normal flight operations but might not 
hold up too well if operated 50 times an hour for 16 hours a day (if this 
were necessary in training) , 

The use of appropriate and well designed simulation is essential to 
cost-effective technical training. Design of the simulation, however, 
must be integrated with the design of the course and must implement the 
course goals, UTien selective practice of crucial job operations and 
appropriate feedback are required in training a simulator must be con- 
sidered. Use uf a GPS provides a reasonable economical simulation 
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capability when a variety of simulations arc required in a training 
program or when low student flow pemits sharing of GPS capability 
among different programs. 
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?UJ^E A\mR THESE QUESTIONS BEFORE TRAINING 

SS.\ _RANK/GRADE YEARS OF SERVICE 

Primary AFSC 
Secondary AFSC 

Have you had any, experience with the A-7 aircraft? Yes No 
I have experience working on equipment on the following aircraft 
,series: 



Are you tanuiiar witn tne Ar^j i^o Kaaar oysLem; 


Vac 
I \zS 






Have you received training on the APQ 126 
Radar System? 


Yes 


No 




If yes, did you learn from: 








(a) Actual equipment trainer 


Yes 


No 


P 


(b) Equipment as installed ^ on the A-7 aircraft 


Yes 


No 


P 


(c) TO and lecture 


Yes 


No 


P 


(d) EC II Simulator 


Yes 


No 


P 


(Please circle P for primary mode of training) 








Have you worked on the APQ 126 Radar System? 


Yes 


No 




Are you qualified to troubleshoot the APQ 126 
Radar System? 


Yes 


No 




Have you worked on other radar systems? 


Yes 


No 





If yes, which_ 
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?:-sSl /vXSCr TiESE c:jzs::c\s following tts/jning 



SSN 

Did ycu like us in,-; this approach to learning? Yes No 

2. Was ti.c learning (a) easy or (b) difficult when using the ECU Sirr.ulator? 

(circle one) (^) Co) 

3. Did you learn the purpose of xl\Q controls? Yes No 

4. Can vcu interpret norr.al versus abnonr^l 

Or.al function} operation? .No 

5. Can you perfonr; the appropriate systea self checks? Yes Xo 

6. Can you isolate malfunctions? Yes No 

7. Will this training enable you to work more ^ 

effectively on the actual equip.T*ent? *cs No 

8. /\s a training person using as a prL^r.ary mode of instruction 

cacl^ of the following, modes 

(a) Actual equipr.ent trainer 

(b) Bc;uipment as installed on A/C 

(c) TO and lecture 

(d) LC II Simulation 

Uhicii mode would be the fastest? (a) (b) (c) (d) (Rank order with 

top choice ffl) 

mode would provide most ^ , ^ , , • u 

effective learning? (a) (b) (c) (d) (Rank order ^>ath 

top choice #1) 

W^ucl mode would make learning . . , r , 

the easiest? (a) (b) (c) (d) (Rank order with 

top choice «1) 

9. Would vou recomiTicnd the use of a simulator such as the EC 11 
siir/ula'tion of the APQ 126 for your training program? 

(a) For the APQ 126 Yes No 

Co) For other systems to be trained Yes No 
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MALFUNCTION MODES 



NUMBER OF REPLACEMENTS TOTAL TIME 
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